The Important Numbers of Testing: 0, 1, and Many

Mark Leighton Fisher on 2008-06-18T10:58:43

Although there are an infinity of numbers to use in software testing, the 3 important numbers are 0, 1, and Many.

0, the number of nothingness, comes into play when you don't have anything. C enshrined 0 as the null pointer, though other languages and systems had represented nothing by a memory address of 0 before C. (There were other representations of null — they make for interesting reading.) Customers without orders, Webpages without links, forests without oak trees — all of these are most easily represented by a 0 inside a computer. Even inside your computer, your programs are not a closed system. You can run out of memory (although Perl eliminates the silly cases of this), you might forget and make a directory unreadable (0 files), a compiler error could skip an allocation statement (0 elves) — the list can go on and on. If you don't consider the case of 0, eventually your software will fail. (Conversely, I once wrote a server in Perl 4 that ran for months at a time because I did extensively consider and test the case of 0.)

1, the first number, is seen when you only have one of something, an idea so common that it becomes the Singleton pattern in languages that need a special representation of one and only one instantiation of a class. With 1 of something, everything has to be instantiated, but you don't have the problems of multiple copies of the item in question. If the item is part of a collection of items, I have occasionally seen defects where the collection is not allocated if there is only 1 item. It is probably an artifact of my coding style, but I don't see many defects in my code specific to 1 and only 1 item. When the common cases are 0 and many, I have seen code that fails to work on all of the edge cases of 1 and only 1 item.

"Many" often just means "more than 1". Usually, your code does nothing different for the 472nd item than it does for the 2nd item. There are times, though, when code(2nd) != code(472nd) (3-column display code comes to mind here). Handling many items involves sizing their containers appropriately. Almost any allocation algorithm can give you space for 1 item — only correctly constructed allocation algorithms will always yield the right number of places to contain your items. The familiar fence-post error of array management is but one example of a failed allocation algorithm (and failure to properly test for many items).

defined() is the special case of accessing an item before it is initialized. A real-world case is a restaurant without any customers. Attempts to access any customer data will only find undefined values. Undefined values can occur when you grab large blocks of data for performance reasons — the example restaurant and all its customers from before — where you grab so much data in one fell swoop that not all of it is initialized. Incomplete data is not defined. A customer without a cellphone (or without a landline phone) would have an undefined value for that phone field. A broken tire pressure sensor could yield an undefined value when read. Sometimes you can just ignore undefined values, but other times you have to explicitly handle them (think running sums or some statistical operations).

Although you may have other special numbers to test, you will likely have to test at least 0, 1, and many. The multiples of many, the somethingness of 1, and the nothingness of 0 will need to be tested to ensure adequate test coverage of your code.